Dual-Level Parallelism Exploitation with OpenMP in Coastal Ocean Circulation Modeling

Two alternative dual-level parallel implementations of the Multiblock Grid Princeton Ocean Model (MGPOM) are compared in this paper. The first one combines the use of two programming paradigms: message passing with the Message Passing Interface (MPI) and shared memory with OpenMP (version called MPI-OpenMP); the second uses only OpenMP (version called OpenMP-Only). MGPOM is a multiblock grid code that enables the exploitation of two levels of parallelism. The MPI-OpenMP implementation uses MPI to parallelize computations by assigning each grid block to a unique MPI process. Since not all grid blocks are of the same size, the workload between processes varies. OpenMP is used within each MPI process to improve load balance. The alternative OpenMPOnly implementation uses some extensions proposed to OpenMP that defines thread groups in order to efficiently exploit the available two levels of parallelism. These extensions are supported by a research OpenMP compiler named NanosCompiler. Performance results of the two implementations from the MGPOM code on a 20-block grid for the Arabian Gulf simulation demonstrate the efficacy of the OpenMP-Only versions of the code. The simplicity of the OpenMP implementation as well as the possibility of using and simply defining policies to dynamically change the allocation of OpenMP threads to the two levels of parallelism is the main result of this study and suggests to consider this alternative for the parallelization of future applications.